Databases of images are used in a wide variety of applications, such as medical imaging, security analysis of luggage, satellite observations of earth, and so on. In the particular context of medical imaging, the images may be in two-dimensional or three-dimensional form, and may be acquired through various modalities, such as MRI, X-ray, and so on. These images can then be used for a range of purposes, including detecting and monitory disease or abnormalities, preparing and performing interventions, such as surgery, and so on.
An image may be processed to perform segmentation and/or annotation. For example, the segmentation may determine which parts of the image relate to a particular organ (or organs), while annotation might label certain features that are visible in the image. In neuroimaging, pathological classification techniques have mostly relied on such features extracted from MRI images, either using voxel-based features [16,17] or features estimated from segmented structures or regions-of-interest (ROI) known to be affected by certain pathologies, like gray matter density [18], volume [19] or shape [20]. Despite remarkable success of the above methods, there are shortcomings resulting from the inherent complexity of the problem. Voxel-based methods rely on the ability to map all the pathological and non-pathological subjects onto a common space of comparison, called a groupwise space. This correspondence is a highly ill-posed problem which cannot be uniquely determined by intensity-based image attributes and where an exact anatomical correspondence may not exist at all due to anatomical variability across subjects. For example, it may not be possible to perfectly map subjects with two different cortical folding patterns through a biologically meaningful transformation. Thus, large morphological differences between an individual and the groupwise space can result in a significant amount of residual information that the transformation does not capture [21]. This residual mismatch between subjects acts as a morphological confound [22], reducing the statistical power of voxel-based techniques. On the other side, ROI-based techniques do not use groupwise spaces for comparison and thus do not have the same mapping limitation. However, they require selecting and accurately segmenting the ROI, a process that can be time-consuming, prone to error, and requiring a priori hypotheses on a relevant ROI. Moreover, ROI-based techniques effectively perform feature reduction, discarding potentially useful features and resulting in lower performance for pathology classification [23].
Recently, a third class of techniques has emerged based on the concept of label propagation and fusion. This class of techniques typically uses a set of anatomical images (atlas) with an associated voxel-wise label map. Using image registration techniques, one can propagate the atlases and their corresponding label maps to a new unseen image. After alignment, the most similar atlases and labels (according to a similarity metric) are then used in the fusion step in order to obtain an optimal label map for the new unseen image. Typically, the previously described label maps represent a structural brain parcellation [24,25], although this framework can be extended to any type of information, i.e. probabilistic segmentations, population atlases and pathological classifications. Recent work by [26] explored the use of a label fusion framework in order to pathologically grade anatomical structures. Despite the many limitations of this framework, a high degree of accuracy for “AD vs. control” classification was obtained, namely a success rate of 90%, which is competitive with state-of-art machine learning approaches.
However, structural and anatomical information is only available for a small subset of image data that is available to researchers in various image databases. Likewise, clinical data (e.g. information about the subject of the image and his/her medical condition) is also rarely available. Unfortunately, providing this additional information, such as manual parcelations and/or segmentations, locations of anatomical landmarks, tissue priors, pathological classification of the subject, and so on, requires large amounts of human interaction, usually from a skilled professional. It is therefore not feasible in practice to provide full information about the images in many databases through manual interaction.
Attempts have therefore been made in the context of medical imaging to extrapolate and propagate this supplementary information across morphologically dissimilar image datasets in a coherent and error-free manner. In neuro-image analysis, one example of such information propagation is multi-atlas segmentation. Many researchers have shown that propagating structural parcelations from multiple sources, by mapping them to new unseen data using image registration and then fusing the candidate parcelations, provides a good estimation of the true underlying parcelation [1, 2]. However, in the case of a limited and morphologically clustered source of information, like the 30 young control subjects with an associated parcelation of 83 key brain areas provided by Hammers et al. [3], structural parcelation propagation can be problematic. As these parcelations are defined only on young controls with normal anatomy, it is non-trivial to map this information directly to morphologically dissimilar and pathological subjects [4, 5] without introducing large errors.
More recently, Wolz et al. [6] introduced the LEAP approach (learning embeddings for atlas propagation) for brain segmentation (see also WO2011/039515), in particular, for segmentation propagation of the hippocampus. In LEAP, a low-dimensional representation of the data is used to find morphologically similar datasets. This morphological similarity is then used to gradually diffuse the segmentation of the brain from the 30 Hammers atlases [3] to pathological Alzheimer's diseased patients via morphologically similar intermediate datasets, greatly increasing the segmentation accuracy. Accordingly, the available segmentations are gradually propagated and transferred between morphologically disparate subjects via multiple morphologically intermediate images. However, this method has still some limitations regarding its computational complexity, its restriction to global morphometric characterization and the limited set of features it can use. In addition, as the region-of-interest size increases, the morphological embedding becomes less localised, potentially resulting in a decrease in performance. A similar framework, but for geodesic image registration, was also introduced by Hamm et al. [7] with the GRAM (geodesic registration on anatomical manifolds) method. This family of step-wise propagation algorithms will become increasingly relevant as the availability of larger and larger image databases continues to drive the need for automatically propagating segmentation, annotations, and other such additional information.